Eye tracking system with biometric identification

ABSTRACT

A method for measuring eye tracking in a patient and capturing ocular biometric identifying information from the patient may involve: displaying a video to the patient on a stimulus screen of an eye tracking and biometric identification system; tracking movement of the patient&#39;s eyes during display of the video via an eye tracking camera of the eye tracking and biometric identification system; capturing ocular biometric identifying information from the patient with an ocular biometric capture device of the eye tracking and biometric identification system; generating, with a computer processor of the eye tracking and biometric identification system, a score representing an ability of the patient&#39;s eyes to track the video; and confirming, with the computer processor, an identity of the patient, based on the captured ocular biometric identifying information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/786,759 filed on Oct. 18, 2017 entitled, “EYE TRACKING SYSTEM WITHBIOMETRIC IDENTIFICATION,” which claims priority to U.S. ProvisionalPatent Application Ser. No. 62/410,754, entitled “Apparatus forBiometric Identification Within An Eye-Tracking Apparatus ForNeuro-Diagnosis,” filed Oct. 20, 2016, the full disclosure of which ishereby incorporated by reference.

This application also hereby incorporates by reference: U.S. Pat. No.9,642,522; U.S. Patent Application Pub. Nos. 2016/0278716, 2017/0172408and 2018/0092531; and U.S. Patent Application Ser. No. 62/558,069,titled “Eye Tracking System,” filed Sep. 13, 2017. The above-listedpatents and applications may be referred to collectively below as “TheIncorporated References.” The above-listed patents and applications, aswell as all publications, patent applications, patents and otherreference material mentioned in this application, are herebyincorporated by reference in their entirety.

TECHNICAL FIELD

This application is directed to medical devices, systems and methods.More specifically, the application is directed to devices, systems andmethods for providing eye tracking and biometric identification.

BACKGROUND OF THE INVENTION

Many central nervous system injuries and abnormalities can bechallenging to diagnose and localize within the nervous system. Theassignee of the present application has developed methods and systemsthat use eye tracking measurement to help diagnose and/or localize anumber of different central nervous system injuries and abnormalities,such as but not limited to increased intracranial pressure (ICP),concussion, traumatic brain injury (TBI), reduced or impaired cranialnerve function, and the like. Some of these methods and systems aredescribed in the Incorporated References.

BRIEF SUMMARY

The present application is directed to a system and method that combineseye tracking capabilities with ocular biometric identification of anindividual. Using the system and method, biometric identification may beperformed before or during an eye tracking session. Identification ofthe patient may be performed by a cloud-based biometric service. Invarious embodiments, the biometric identification component of thesystem may be incorporated into, or added to, any suitable eye trackingsystem, such as but not limited to the eye tracking systems described inthe Incorporated References.

These and other aspects and embodiments are described in greater detailbelow, in reference to the attached drawing figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C are patient-facing, side, and technician-facing views,respectively, of a system for measuring eye tracking, according to oneembodiment;

FIG. 2 is side-view diagram of an eye tracking system that includes anocular capture device for biometric identification, according to oneembodiment; and

FIG. 3 is a flow diagram, illustrating a method for eye tracking andbiometric identification, according to one embodiment.

DETAILED DESCRIPTION OF THE INVENTION

The assignee of the present patent application has developed eyetracking devices, systems and methods that may be used to track themovement of a patient's pupils over time as they follow a video movingacross a screen and measure metrics such as distance traveled over timeand the ability to coordinate eye movements. Cranial Nerve III and VIpalsies, for example, may be identified using this system and method,which includes at least one computer processor that employs an algorithmto perform diagnostic calculations. These devices, systems and methodsare described more fully in the Incorporated References. The presentapplication uses the same technology, method and algorithm to provide anon-invasive eye tracking methodology to measure ICP, and combines thattechnology and methodology with a device and method for biometricidentification of the patient.

Referring to FIGS. 1A-1C, an eye tracking, biometric identification anddiagnostic system 10 is illustrated, according to one embodiment. Theeye tracking portion of system 10 is described in further detail in U.S.Provisional Patent Application Ser. No. 62/558,069, which was previouslyincorporated by reference. System 10 is used to track a patient's eyemovement and diagnose one or more eye movement abnormalities. In someembodiments, system 10 includes at least one processor, which mayfurther analyze data related to the eye movement abnormalities to helpdiagnose and/or localize a neurological injury or abnormality, such asbut not limited to increased ICP, concussion, TBI, or reduced orimpaired cranial nerve function. In use, a stimulus video is presentedon system's 10 LCD stimulus screen 12, and the patient's eye movement istracked over time by an infrared eye tracking camera 14. Fixedorientation of the patient's head, relative to the stimulus screen 12and camera 14, is ensured by a head rest assembly 18, which adjusts to aseated or supine patient. System 10 is operated from a touchscreeninterface 26. System 10 is coupled together via a wheeled chassis 20suitable for wheeling to the examination location.

Chassis 20 includes two main parts: a main column 28 supported by a base22, and a head rest assembly 18. Head rest assembly 18 is supported onan arm 24, which can be raised and lowered using an electrically-drivenelevator in main column 28, controlled by an up/down button 38 (FIG.1C). Arm 24 moves up and down through a vertical slot 29 (FIG. 1A) inmain column 28. Four locking castors 32 in base 22 allow the entire unitto be moved, with base 32 being sized to fit underneath a patient bed orgurney. A handle 30 on main column 28 is used to push and/or pull system10 into place.

In one embodiment, main column 28 houses two computers, a power supply,the elevator mechanism, an isolation transformer, and other electricalparts, none of which is visible in the figures. Operator touchscreeninterface 26 (also called “operator console 26” herein) is located onmain column 28.

Head rest assembly 18 includes a chin rest 34 and a forehead rest 36, tostabilize the patient's head, stimulus screen 12, an optical mirror 16used to fold the optical path allowing for more compact casing, and ahigh-speed eye tracking camera 14. The entire head rest assembly 18 canbe rotated in the horizontal plane 90 degrees in either direction, for atotal horizontal rotation of 180 degrees, and up to 90 degrees in thevertical direction downward to accommodate supine patients. In oneembodiment, there are several discrete positions within the verticalrotation where head rest assembly 18 locks into place. Buttons 40 on theback of head rest assembly 18 activate solenoids, so assembly 18 can berotated vertically and then locked.

A standard 110-volt medical grade cord may provide power to system's 10elevator mechanism and a 400-watt power supply. The power supplyprovides regulated DC power to the computers, as well as the solenoidcontrols in head rest assembly 18.

System 10 includes two computers, which are housed in main column 28 ofchassis 20 and thus not visible in the figures. A camera computer, whichmay be provided by the same manufacturer as the manufacturer of camera14, may run the real-time software for camera 14 under a real-timeoperating system. It detects eye motion events, such as saccades,blinks, and fixations, and computes the gaze coordinates for each eye at500 Hz, storing the raw data until it is needed by the application. Theapplication computer may be a small form-factor PC that runs a systemapplication for system 10. The system application provides the userinterface, controls the logic flow, displays the stimulus video,processes the raw data from the camera computer, and stores results inpersistent storage.

The user interacts with the system application through touchscreeninterface 26. Stimulus screen 12 (the second monitor on system 10)displays the stimulus media to the patient. Two built-in speakersprovide the audio for the stimulus media.

In some embodiments, the processor(s) in system 10 is configured togenerate a score describing a patient's eye tracking ability. Forexample, in one embodiment, system 10 generates a score ranging from0-20, where the score is interpreted as a binary classification for eyemovement abnormalities, and where anything equal to or greater than 10is a positive result (abnormality present) and everything below 10 isnegative (no abnormality). The system's 10 operating algorithmidentifies eye tracking abnormalities and computes the score.

In one embodiment, eye tracking camera 14 is an EyeLink 1000 Plus USB(SR Research, Ottawa, Canada) and is used to capture the eye movementsof the patient. Camera 14 captures 500 frames of gaze data per secondfor each eye, with an average accuracy of 0.25 to 0.5 degrees. Theilluminators are infrared, and it uses dark pupil eye tracking, in whichthe infrared sources are offset from camera 14. This technique typicallyprovides better results across ethnicities and varied lightingconditions. The gaze tracking ranges up to 32 degrees horizontally and25 degrees vertically. The distance between the subject's eyes and thecamera is 52 cm. The specifications for camera 14, as provided by thevendor, are shown in Table 1.

TABLE 1 EyeLink Camera Specifications Spec Description Average accuracyof gaze Down to 0.15 degrees (0.25 degrees to 0.5 coordinate datadegrees typical) sampling rate Binocular: 250, 500 hz End-to-end sampledelay m < 1.8 msec, sd < 0.6 msec @ 1000 hz Blink/occlusion recovery m <1.8 msec, sd < 0.6 msec @ 1000 hz Spatial resolution <0.01 degrees rmsEye tracking principle Dark pupil - corneal reflection Pupil detectionmodels Centroid or ellipse fitting Pupil size resolution 0.2% ofdiameter Gaze tracking range 32 degrees horizontally, 25 degreesvertically Allowed head movements ±25 mm horizontal or vertical6, ±10without accuracy reduction mm depth Optimal camera-eye distance Between40-70 cm Glasses compatibility The user must remove their glasses to usethe system On-line event parsing Fixation/saccade/blink/fixation updateReal-time operator feedback Eye position cursor or position traces.Camera images and tracking status.

Eye Tracking Computer

As mentioned above, in one embodiment, camera 14 is driven by adedicated real-time computer running the QNX operating system. Thespecifications for this eye tracking computer are shown in Table 2.

TABLE 2 EyeLink Computer Specifications Spec Description Design Picoform factor; 8-layer SBC PCB size: 100 mm × 72 mm Embedded CPU IntelBraswell SoC CPU Memory Onboard 4 GB unbuffered DDR3L 1600 MHz DRAMExpansion slot 1 full-size mini-PCIE slot Storage SATA III port M.2M-key 2242 slot LAN chip Integrated with Intel i211 AT PCI-E Gigabit LANchip Support fast Ethernet LAN function of providing 10/100/1000 MbpsEthernet data transfer rate Audio chip Realtek ALC662 2-CH HD audiocodec integrated Audio driver and utility included BIOS 64 Mbit flashROM Rear I/O 12 V DC-in power jack USB 3.0 port (2) Display port RJ-45LAN port Internal I/O 2-pin internal 12 V DC-in power connector SATApower-out connector Front panel audio header 9-pin USB 2.0 header Serialport header (2) Front panel header LAN LED activity header LVDS headerLVDS inverter

System Application Computer

As mentioned above, in one embodiment, the system application runs on amini-ITX board running Windows 10 Pro, configured as a kiosk device. Thespecifications are shown in Table 3.

TABLE 3 System Application Computer Specifications Spec DescriptionDesign Mini ITX form factor CPU Intel i7 (speed, etc TBD) Chipset IntelH170 Memory 16 GB dual channel DDR4 Expansion slot PCI Express 3.0 × 16slot Vertical half-size mini-PCI Express slot Graphics Intel HDgraphics, dual graphics output, DVI/HDMI max resolution to 4K × 2KAdditional Invidia GeForce 210, DVI/VGA/DisplayPort max graphicsresolution to 2560 × 1600 Audio 7.1 CH HD audio LAN Intel 1219V (gigabitLAN) Realtek RTL8111H (gigabit LAN) Rear I/O PS/2 mouse/keyboard portDVI port HDMI port USB 2.0 ports (2) USB 3.0 ports (6) RJ-45 LAN ports(2) HD audio jack Storage SATA 3 256 GB SSD BIOS 128 MB AMI UEFI legalBIOS Certifications FCC, CE, WHQL

Stimulus Display

Stimulus screen 12, according to one embodiment, is used to display avideo that may last any suitable length of time, such as 220 seconds inone embodiment. In one embodiment, the only purpose of stimulus screen12 is to display the visual stimulus. The video may be one of severalpre-determined videos. These videos may include music videos, clips fromchildren's movies, sports clips, talent performances, “reality TV”clips, etc. The choice of videos may be designed to appeal to a broadgroup of subjects. Users of the device may choose which video to displayor may ask the patient which one they would like to watch. Additionalmedia selections may be downloaded via a UBS drive, for example. In oneembodiment, the video aperture is square, with each side beingapproximately ¼ the width of the visible display. The trajectory of thedisplayed video around stimulus screen 12 follows a predefined discretepath, such as 5 cycles along the perimeter of stimulus screen 12 with avelocity of 10 seconds per side, according to one embodiment. In oneembodiment, stimulus screen 12 is an GeChic 1303 monitor, with thespecifications shown below in Table 4.

TABLE 4 Stimulus Screen Specifications Spec Description Aspect ratio1.78:1 Maximum resolution 1920 × 1080 Screen size 13.3 inches Displaytype LED Viewing angle 89°/89°/89°/89° Contrast ratio 700:1 Power input5 V, 2.0 A

Touchscreen Interface

Touchscreen interface 26 (which may also be referred to as an “operatorconsole” or simply “touchscreen”) is used by the technician to interactwith the system application. In the pictured embodiment, touchscreeninterface 26 includes only a touch screen display, meaning that there isno keyboard or other input device. Of course, alternative embodimentsmay include a keyboard or other input device(s). In one embodiment,touchscreen interface 26 may be a Mimo UM-1080CH-G, with thespecifications set forth below in Table 5.

TABLE 5 Touchscreen Interface Specifications Spec Description Capacitivetouchscreen Yes Maximum resolution 1280 × 800 Screen size 10.1 inchesViewing angle 170° × 170° Contrast ratio 800:1 Power input 6 W

Head Rest Assembly

Chin rest 34 and forehead rest 36 are used to stabilize the user's headand maintain appropriate distance from stimulus screen 12 during eyetracking. Chin rest 34 may be made from the non-toxic, non-hazardousbiodegradable plastic Bakelite resin(polyoxybenzylmethylenglycolanhydride), and forehead rest 36 may beconstructed from aluminum covered with a thin EPDM (ethylene propylenediene terpolymer) foam pad blended with neoprene and SBR(styrene-butadiene rubber) pad with closed-cell construction, to resistliquid, solid, and gas absorbance. Both surfaces may be wiped usingsterile alcohol swabs before and after each use.

System Calibration

The calibration information below in Table 6 applies to the componentsof system, according to one embodiment.

TABLE 6 System Calibration Component Calibration notes Eye tracking Afocus knob on the bottom of the unit adjusts focus. camera 14 Once it isset, it generally does not need any adjustment unless the knob isaccidentally jarred. The user guide provides instructions for adjustingfocus. Gaze point calibration is not required. System 10 uses camera's14 built-in default calibration for calculating gaze points from pupildetection and corneal reflection measurements. Stimulus Stimulus display12 is surrounded by bezels to reduce the display 12 size of display 12to effectively be 4:3 instead of 1.78:1. The software uses calibrationinformation stored in the device configuration file to determine whatpart of the display is actually visible to the user. The configuration,once set, does not change since the bezels and display are fixed. Theseconfiguration parameters will be set at the factory before being shippedto the end user. Operator The aspect ratio will be set when the unit isassembled. console 26 No additional calibration by the user is needed.Computers The computers perform a boot sequence upon startup that willrun diagnostic procedures to ensure correct operation. No additionalcalibration of the computers is necessary. Head rest At assembly time,the head rest assembly 18 rotation assembly 18 limits are set and fixedinto place. No additional rotation calibration is needed. Chin rest 34The user should use an alcohol wipe before and after and forehead eachpatient in order to sterilize the parts of the device rest 36 that comein contact with the patient. Instructions for this are included in theuser guide. Elevator At assembly time, the elevator height limits areset and fixed into place. No additional calibration is needed. Theelevator is not expected to require any maintenance during the normallifetime of the device. Optical Dust or dirt may accumulate in thecamera/mirror bay mirror 16 of head rest assembly 18. User guideinstructions provide information about how to remove the debris usingcompressed air if needed.

Principle of Operation

System 10 measures a patient's eye tracking while watching a video movearound stimulus screen 12 and then analyzes the data from the eyetracking measurements, using an algorithm, to extract clinicallyrelevant eye measures by using temporal assessment. The patient watchesa video moving inside an aperture with a set trajectory for 220 seconds(in one embodiment) at a fixed distance from stimulus screen 12. Theposition of each pupil is recorded over time elapsed, as the videotravels on its time course, enabling detection of impaired ability tomove the eyes relative to time and therefore relative to each other. Thealgorithm inputs are measurements of the individual (left and right)eye-movements, averaged over the 5 cycles that the eyes move whilewatching the 220-second video that plays in an aperture moving aroundscreen 12. In one embodiment, the algorithm output is a “BOX Score,”calculated by multiplying multiple constants with different individualparameters, and summing those factors.

System Computer Processing Overview

Central to the operation of system 10 is how the software processes rawgaze data from the eye tracking camera and calculates a BOX score. Anoverview of this process is outlined below.

-   -   1. During a tracking, system 10 collects 220 seconds of        binocular gaze data at 500 Hz as the patient watches the video        stimulus go around screen 12 five times. The trajectory is as        follows:        -   a. The stimulus starts in the upper left corner and remains            stationary for 10 seconds.        -   b. The stimulus moves around the outer edges of the stimulus            screen in a clockwise fashion, taking ten seconds for each            side or 40 second for each cycle. The stimulus makes five            cycles plus one extra side along the top for a total of 220            seconds.    -   2. The first and last ten seconds of data are discarded.    -   3. The processing first removes blink data. These data become        “NaN's” (not a number) in the processing and are ignored in all        subsequent processing.    -   4. Next, the gaze data are normalized, and several dozen metrics        are computed from the normalized data, which characterize the        patient's tracking. Some of the metrics are conjugate values        that evaluate the differences between the left and right eye.    -   5. The most correlated metrics are then fed into a polynomial        formula that produces a BOX score between 0 and 20. Scores less        than 10 are considered normal.    -   6. The algorithm also computes a quality score (1 to 10 with 10        being the best) based on the percentage of NaN's that were in        the raw data.

Referring now to FIG. 2, a diagrammatic representation of a combined eyetracking and biometric identification system 100 is provided. In thisembodiment, system 100 includes an ocular biometric capture device 110,a stimulus monitor 112, an eye tracking camera 114, a central processingunit (CPU) 116, a chin rest 134, a forehead rest 136, and a biometricmatching engine and database 120. Any or all of the details regardingthe eye tracking features and components of system 100 may be the sameas, or similar to, those described above in relation to system 10 andFIGS. 1A-1C. During use, the patient's face is stabilized in chin rest134 and forehead rest 136, at a precise distance from stimulus monitor112. Eye tracking camera 114 is positioned just below stimulus monitor112 and captures images from the eyes. Ocular biometric capture device110 is positioned just above stimulus monitor 112 and captures biometricdata from the eyes. In an alternative embodiment, eye tracking camera114 may be configured to capture ocular biometric data from the patient,thus eliminating the need for a separate ocular capture device 110.Central processing unit 116 controls the display of the stimulus media,eye tracking camera 114 and ocular biometric capture device 110. Centralprocessing unit 116 also communicates with cloud-based biometricmatching engine and database 120.

Referring now to FIG. 3, a flow diagram of a combined eye tracking andbiometric identification method 200 is provided. In the illustratedembodiment, biometric identification and eye tracking are shown as ifperformed in parallel. This may be the case in some embodiments, butaccording to various embodiments, it is possible to perform biometricidentification before, during and/or after eye tracking. In other words,method 200 is not limited to one particular timing or sequence.

In the illustrated embodiment, at the beginning of the procedure 201,two threads of execution are initiated, one that begins an ocularbiometric data capture method 202 and one that begins an eye trackingmethod 212. Eye tracking method 212 may be the same as, or similar to,the methods described above. In FIG. 3, the details of eye trackingmethod 212 are not illustrated and instead are shown simply aseye-tracking session complete 214, which is typically signaled by theend of the stimulus video. Biometric method 202 involves capturing anocular biometric sample and sending it to a cloud-based biometricidentification service 204, for example including matching engine anddatabase 120 (FIG. 2). The service attempts to match the biometricsample with a record in the biometric data 206. If a match is found 208,biometric method 202 associates the patient with the currenteye-tracking session, and waits until the eye tracking session iscomplete 214. If a match is not found 210 and the eye tracking is notcomplete, another biometric sample is taken 202 and sent to thecloud-based service 204. This process continues until either a match isfound or the eye-tracking session is complete. Sending multiplebiometric samples to the cloud-based service decreases the chance of theultimate result being that no match is found, even though the patient isenrolled. If the eye tracking session completes without a match beingfound, the patient's demographic data is requested, linked to thebiometric sample(s), and sent to the cloud-based service as a newenrollment 218. This method 200 is just one embodiment, and variationsmay be made without departing from the scope of the disclosure. Forexample, alternative embodiments may include fewer steps, greaternumbers of steps and/or different ordering of steps.

The foregoing is believed to be a complete and accurate description ofvarious embodiments of a system and method for assessing glaucoma in apatient. The description is of embodiments only, however, and is notmeant to limit the scope of the invention set forth in the claims.

We claim:
 1. A system for measuring eye tracking in a patient andcapturing ocular biometric identifying information from the patient, thesystem comprising: a stimulus screen for displaying a video to thepatient; an eye tracking camera; at least one head rest member forstabilizing the patient's head, relative to the stimulus screen; anocular biometric capture device for capturing the ocular biometricidentifying information from the patient; and a system computer housedin the main column for controlling the stimulus screen, data processingand other functions of the system.